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Summary 



Formative assessment policies, programs, 
and practices in the Southwest Region 



Formative assessments help educators 
target instructional practices to meet 
specific student needs and monitor and 
support student progress toward valued 
state learning outcomes. Policies and 
programs in the five Southwest Region 
states suggest a range of strategies to 
support the development and use of 
formative assessments. 

This report describes state formative assess- 
ment policies, programs, and practices in the 
five states covered by the Southwest Regional 
Educational Laboratory: Arkansas, Louisiana, 
New Mexico, Oklahoma, and Texas, forma- 
tive assessments, unlike summative assess- 
ments, yield descriptive data — not judgments. 
They are used diagnostically and to improve 
instruction — not, for example, to assign end- 
of-course grades (Wiggins, 1998). 

Such distinctions are reflected in a definition 
of formative assessment recently adopted by 
the Council of Chief State School Officers: 

“An assessment is formative to the extent that 
information from the assessment is used, 
during the instructional segment in which 
the assessment occurred, to adjust instruction 
with the intent of better meeting the needs of 
the students assessed” (Popham, 2006). This 
definition was used to guide the collection of 
formative assessment data for this study. 



A systematic examination uncovered dispari- 
ties in how states define formative assessment. 
Arkansas, Louisiana, and New Mexico have 
formal definitions — but the definitions differ, 
suggesting that Southwest Region states view 
formative assessment through slightly differ- 
ent lenses. (No formal definitions were discov- 
ered for Oklahoma or Texas.) 

Researchers found no study that specifically 
explored the effects of formative assessment 
policy on local practice. But related research 
on school reform (Darling-Hammond & 
McLaughlin, 1995), assessment reform (Chu- 
dowsky & Pellegrino, 2003; Stiggins, 2002), 
and accountability (Goodwin, Englert, & 
Cicchinelli, 2003) suggests that explicitly com- 
municating a clear, consistent message about 
effective practices facilitates cross-district 
consistency with the state’s education goals. 

Three Southwest Region states have been most 
explicit in communicating their intent for 
formative assessment to stakeholders through 
state policies or mandates, state-supported 
programs or products, or allotments of ser- 
vices and resources to districts: 

• Arkansas’ House Bill 2253, although cur- 
rently withdrawn pending further study, 
calls for a two-year pilot using formative 
assessments statewide. 
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• Louisiana’s Enhanced Assessments of 
Grade-Level Expectations gives teachers 
access to an online pool of custom items 
aligned to state standards, with additional 
tools for individualizing instruction. 

• New Mexico’s Consumer Guide (New 
Mexico Public Education Department, 
2006b) evaluates vendors’ tests and judges 
their appropriateness for formative use by 
state districts. 

Each state’s position appears intended to fit 
coherently within its existing comprehensive 
assessment system and state context. 

Substantial variability emerged in how much 
states regulate the development or use of 
formative assessments at state and local levels. 
Researchers found a range of state laws, with 
formal state directives only in Arkansas, 
Oklahoma, and Texas. State education agency 
policies also vary across the Southwest Re- 
gion. In Arkansas the Academic Improvement 
Plan specifies that districts must implement a 
formative assessment component, but districts 
are responsible for selecting and purchas- 
ing the tool for this (Arkansas Department 
of Education, 2006a). Similarly, Oklahoma 
requires end-of-course test data to be reported 
to districts for formative use, but does not sup- 
ply districts with strategies for reaching this 
goal. In Texas the Technology Immersion Pilot 
provides a platform for administering online 
diagnostic assessments, though the test items 
are to be provided by each district (Texas 
Education Agency, 2006b). Louisiana’s guid- 
ance is most direct: a state grant allows the 



Louisiana Department of Education to provide 
all districts with an online formative assess- 
ment system, including both a pool of custom 
items (aligned to state standards) and training 
in collecting and reporting data for formative 
purposes. 

Variability also was observed in the support 
provided to districts for formative assessment, 
such as professional development opportuni- 
ties, resources, and product endorsements. 
Researchers found evidence of five different 
state-sponsored professional development op- 
portunities related to formative assessment for 
Arkansas educators, but none for New Mexico 
educators. Oklahoma was the only state for 
which researchers could find no evidence of 
state resource allocation for formative assess- 
ment. But Oklahoma, along with Arkansas 
and Texas, provides districts with endorse- 
ments for particular programs or products 
related to formative assessment. 

Overall, Arkansas, Louisiana, and Texas appear 
to have developed multitiered or multifaceted 
strategies for supporting formative assess- 
ment at the state and local levels. For example, 
Arkansas provides state guidance and sup- 
port through regulations, programming, and 
professional development opportunities for 
teachers through resource allocation. In con- 
trast, New Mexico focuses its support primarily 
on one state initiative, its Consumer Guide. In 
Oklahoma limited evidence emerged on state 
policies and programs, but a range of district 
formative assessment practices was found. 
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Formative 
assessments help 
educators target 
instructional 
practices to 
meet specific 
student needs 
and monitor 
and support 
student progress 
toward valued 
state learning 
outcomes. 
Policies and 
programs in the 
five Southwest 
Region states 
suggest a range 
of strategies 
to support the 
development and 
use of formative 
assessments. 



OVERVIEW 

Under the No Child Left Behind Act of 2001 
states are held accountable for the performance 
of all students. Within this context results from 
formative assessments could provide timely and 
descriptive information about students to help 
teachers plan for and deliver effective individu- 
alized instruction. The ability to do this in all 
schools statewide can be facilitated by state policy 
or programming guidance and state resources for 
materials, professional development, and financial 
support. 

This report presents findings from a study of 
formative assessment policies and programs in 
Arkansas, Louisiana, New Mexico, Oklahoma, 
and Texas, the five states covered by the South- 
west Regional Educational Laboratory (see box 1 
for state summary statistics that provide con- 
text for the report findings). Because the study 
was initiated in response to regional interest in 
formative assessment, the types of evidence col- 
lected include information about how each state 
supports the development and use of formative 
assessments at the state and district levels. De- 
scriptive in nature, the report provides a detailed 
portrait of state policies and programs related to 
formative assessment. 

Texas clearly stands out in overall size, with more 
than six times as many students enrolled in nearly 
five times as many schools as the next largest state. 
Among the other states Arkansas and New Mexico 
are more comparable in size, as are Louisiana and 
Oklahoma. Only 14 percentage points separate 
the state with the highest percentage of students 
eligible for free or reduced-price lunch (Louisiana, 
62 percent) from the state with the lowest percent- 
age (Texas, 48 percent). New Mexico identifies 
the highest percentages of students with disabili- 
ties (20 percent) and English language learners 
(19 percent). Arkansas and Texas identify only 
12 percent of their students as having disabilities, 
and Arkansas identifies only 4 percent and Loui- 
siana only 2 percent of their K-12 populations as 
English language learners. 
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BOX 1 

State characteristics 

The table to the right contextualizes 
the study’s findings and promotes 
meaningful cross-case comparisons. 
It presents summary statistics that 
describe characteristics of the K-12 
school populations in Arkansas, Lou- 
isiana, New Mexico, Oklahoma, and 
Texas, such as the number of students 
served and percentages of English 
language learners, students with dis- 
abilities, and students eligible for free 
or reduced-price lunch. Appendix A 
describes the characteristics of each 
state’s K-12 student population in 
greater detail. 



Southwest Region state summary statistics 



Statistic 


Arkansas 


Louisiana 


New 

Mexico 


Oklahoma 


Texas 


K-12 student 
enrollment 


463,115 


724,281 


326,102 


629,476 


4,405,215 


Number of public 
schools 


1,158 


1,541 


842 


1,787 


8,746 


Students eligible 
for free or reduced- 
price lunch (percent) 


52 


62 


58 


54 


48 


Students with 
disabilities (percent) 


12 


14 


20 


15 


12 


English language 
learners (percent) 


4 


2 


19 


7 


16 



Source: U.S. Department of Education, 2006; National Office for Research, Measurement, and Evalua- 
tion, 2005 (Arkansas); Louisiana Department of Education, 2005 (Louisiana); New Mexico Public Educa- 
tion Department, 2005; Oklahoma State Department of Education, 2005 (Oklahoma); Texas Education 
Agency, 2005 (Texas). 



Formative assessments, unlike summative assess- 
ments, yield descriptive data — not judgments. 

They are used diagnostically and to improve 
instruction— not, for example, to assign end-of- 
course grades (Wiggins, 1998). Such distinctions 
are reflected in a definition of formative assess- 
ment recently adopted by the Council of Chief 
State School Officers: “An assessment is formative 
to the extent that information from the assessment 
is used, during the instructional segment in which 
the assessment occurred, to adjust instruction 
with the intent of better meeting the needs of the 
students assessed” (Popham, 2006). 

A systematic examination uncovered dispari- 
ties in how states define formative assessment. 
Arkansas, Louisiana, and New Mexico have formal 
definitions— but these definitions differ, suggest- 
ing that Southwest Region states view formative 
assessment through slightly different lenses. The 
definitions are also communicated to stakeholders 
in different ways, through a variety of sources. (No 
formal definitions of formative assessment were 
discovered in Oklahoma or Texas.) 

Researchers found no study that specifically 
explored the effects of formative assessment policy 



on local practice. But related research on school 
reform (Darling-Hammond & McLaughlin, 1995), 
assessment reform (Chudowsky & Pellegrino, 

2003; Stiggins, 2002), and accountability (Good- 
win, Englert, & Cicchinelli, 2003) suggests that ex- 
plicitly communicating a clear, consistent message 
about effective practices facilitates cross-district 
consistency with the state’s education goals. 

Three Southwest Region states have been most 
explicit in communicating their intent for forma- 
tive assessment to stakeholders through state 
policies or mandates, state-supported programs or 
products, or allotments of services and resources 
to districts: 

• Arkansas’ House Bill 2253, although currently 
withdrawn pending further study, calls for a 
two-year pilot using formative assessments 
statewide. 

• Louisiana’s Enhanced Assessments of Grade- 
Level Expectations gives teachers access to an 
online pool of custom items aligned to state 
standards, with additional tools for individu- 
alizing instruction. 
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• New Mexico’s Consumer Guide evaluates 
vendors’ tests and judges their appropriate- 
ness for formative use by state districts (New 
Mexico Public Education Department, 2006b). 

Each state’s position appears intended to fit coher- 
ently within its existing comprehensive assess- 
ment system and state context. 

Substantial variability emerged in how much 
states regulate the development or use of formative 
assessments at state and local levels. Research- 
ers found a range of state laws, with formal state 
directives only in Arkansas, Oklahoma, and Texas. 
State education agency policies also vary across 
the Southwest Region. In Arkansas the Academic 
Improvement Plan mandated by the state’s rules 
governing the Arkansas Comprehensive Testing, 
Assessment, and Accountability Program specifies 
that districts must implement a formative assess- 
ment component, but districts are responsible for 
selecting and purchasing the tool for this (Arkan- 
sas Department of Education, 2006a). Similarly, 
Oklahoma requires end-of-course test data to be 
reported to districts for formative use, but does 
not supply districts with strategies for reaching 
this goal. In Texas the Technology Immersion 
Pilot provides a platform for administering online 
diagnostic assessments, though the test items are 
to be provided by each district (Texas Education 
Agency, 2006b). Louisiana’s guidance is most 
direct: a state grant allows the Louisiana Depart- 
ment of Education to provide all districts with an 
online formative assessment system, including 
both a pool of custom items (aligned to state stan- 
dards) and training in collecting and reporting 
data for formative purposes. 

Variability also was observed in the support 
provided to districts for formative assessment, 
such as professional development opportunities, 
resources, and product endorsements. Researchers 
found evidence of five different state-sponsored 
professional development opportunities related to 
formative assessment for Arkansas educators, but 
none for New Mexico educators. Oklahoma was 
the only state for which researchers could find no 



evidence of state resource allocation for formative 
assessment. But Oklahoma, along with Arkansas 
and Texas, provides districts with endorsements 
for particular programs or products related to 
formative assessment. 



Substantial variability 
emerged in how much 
states regulate the 
development or use of 
formative assessments 
at state and local levels 



Overall, Arkansas, Louisi- 
ana, and Texas appear to 
have developed multi- 
tiered or multifaceted 
strategies for supporting 
formative assessment at 
the state and local levels. 

For example, Arkansas 

provides state guidance and support through regu- 
lations, programming, and professional develop- 
ment opportunities for teachers through resource 
allocation. In contrast, New Mexico focuses its sup- 
port primarily on one state initiative, its Consumer 
Guide (New Mexico Public Education Department, 
2006b). In Oklahoma limited evidence emerged on 
state policies and programs, but a range of district 
formative assessment practices was found. 



This report does not seek to judge or evaluate the 
effectiveness of formative assessment policies or 
practices. The researchers have excluded such 
evaluative considerations because the context for 
state regulation of formative assessment is con- 
founded in three ways: 

• Formative assessment decisionmaking 
has traditionally been the domain of local 
education agencies rather than state educa- 
tion agencies (Shepard, 2000b). Yet with 
high-stakes assessments based on statewide 
content standards, educators increasingly seek 
state guidance in identifying effective tools 
for gauging student progress toward learning 
goals. 



• Because of a dearth of empirical evidence 
about the effectiveness of such policies in 
stimulating effective formative assessment 
practices, states have no framework for 
weighing tradeoffs associated with different 
implementation models. 
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FORMATIVE ASSESSMENT POLICIES, PROGRAMS, AND PRACTICES IN THE SOUTHWEST REGION 



Clarifying the meaning 
and purpose of formative 
assessment and 
describing its current 
status can help state and 
regional decisionmakers 
weigh options for 
implementing and 
supporting promising 
state and district 
formative assessment 
policies, programs, 
and practices 



• Understandings of what consti- 
tutes formative assessment differ 
within the education community, 
and no consensus has emerged 
about best practice in using data 
formatively. Critical questions 
thus remain unanswered about 
the appropriate nature and degree 
of state guidance for formative 
assessment practice and about the 
criteria forjudging or evaluating 
the effectiveness of such policies. 



While the report seeks to respond 
to a specific need in the Southwest 
Region, it also speaks to a broader challenge facing 
states across the nation. Clarifying the meaning 
and purpose of formative assessment and describ- 
ing its current status in five states can help state 
and regional decisionmakers weigh options for 
implementing and supporting promising state and 
district formative assessment policies, programs, 
and practices. Such efforts may make the links 
between state education goals and state and local 
data use more explicit and contribute to greater 
balance and coherence in state comprehensive as- 
sessment systems. 



Because of the absence of empirical evidence on 
the impact of state policies on formative assess- 
ment practice, more research is needed to support 
state decisionmakers seeking to understand how 
state-level guidance may foster effective local for- 
mative assessment practices and ensure cross-dis- 
trict consistency with the state’s education goals. 



DEFINING FORMATIVE ASSESSMENT 

What is formative assessment? A definition re- 
cently adopted by the Council of Chief State School 
Officers was used to guide the collection of forma- 
tive assessment data for each state: 

An assessment is formative to the extent 
that information from the assessment is 
used, during the instructional segment in 



which the assessment occurred, to ad- 
just instruction with the intent of better 
meeting the needs of the students assessed 
(Popham, 2006). 

As the definition suggests, formative assess- 
ments are intended to support learning and help 
target instruction through feedback that informs 
teachers about student progress toward valued 
learning goals (Sadler, 1989). Such assessments 
can help educators measure learning outcomes or 
detect learning gains over short periods of time, 
rather than just after a year of instruction (Black 
and Wiliam, 1998, 2002, 2006; Commission on 
Instructionally Supportive Assessment, 2001; 
National Research Council, 2001a; Wiliam, 2007). 
They can be used both to describe student learn- 
ing and to evaluate instructional methods. Since 
formative assessments aim to improve instruction, 
they tend to target specific, finely grained learning 
goals rather than broad academic standards (Shep- 
ard et al., 2005). Although some locally designed 
interim or benchmark tests may be included in 
the category, formative assessments should be 
distinguished from statewide standardized tests 
of achievement, end-of-course exams, and mini- 
summative benchmark assessments (practice 
tests) intended to prepare students for high-stakes 
accountability tests (Perie, Marion, &Gong, 2007). 

According to a seminal article by Black and Wil- 
iam (1998), the term formative assessment does 
not merely signify how data are used, but also 
refers to a family of related assessment processes. 
Most educators today agree (Olson, 2006; Wiliam, 
2005, 2007). For the purposes of the present study, 
a formative assessment is any assessment that is 
intentionally and purposefully linked to instruc- 
tion, could yield rich diagnostic information about 
students, and is not intended for assigning sum- 
mative or end-of-course grades (Wiggins, 1998). 



THE NEED FOR FORMATIVE ASSESSMENT 

States have several reasons for developing for- 
mative assessment policies and programs and 
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for supporting formative assessment practices. 
Among them are improving learning outcomes 
for all students, promoting effective instructional 
practices across districts, and creating balance in 
the states comprehensive assessment system. 



Improving learning outcomes for all students 

The processes for using data to improve learning 
outcomes are grounded in cognitive development 
theory about how learning occurs and competence 
develops (National Research Council, 2001b). Clear 
communication of learning goals, with specific 
criteria forjudging progress, ensures that teacher 
and learner share a consistent message. Then, 
through an interactive process, teachers build 
on students’ existing knowledge and support, or 
scaffold, learning in small increments by focusing 
on key errors and prescribing the corrective steps 
needed for improvement. 

During this process students are engaged and guided 
in adjusting their conceptual frameworks and 
learning strategies to received feedback (Pellegrino, 
2006; Stiggins, 2002). The feedback reveals gaps 
between learning goals and the student’s current 
status. When the process is functioning optimally, 
the student can see where problems arise and how 
to adjust in order to meet learning goals. The teacher 
can also use the student’s progressive efforts to refine 
instructional methods (Ramaprasad, 1983; Sadler, 
1989). Because both student and teacher share 
responsibility for learning outcomes, they adapt to 
narrow the gap between what the student knows and 
what learning targets still need to be reached. 

For two reasons formative assessment processes 
might be particularly effective for improving 
learning outcomes for low-performing students. 
First, they are associated with a wide range of 
individualized interventions and instructional 
modifications that have proven effective for ad- 
dressing diverse learning needs (Black & Wiliam, 
2002, 2006; Bloom, 1984; Crooks, 1988; Fuchs, 
Fuchs, Mathes, & Simmons, 1997; Natriello, 1987; 
Nyquist, 2003; Wiliam, 2007). Results used for- 
matively have been useful in guiding curricula or 



instruction suited to a particular student popula- 
tion (Boston, 2002; Shepard et al., 2005). 

Second, formative assessment processes provide 
specific information about key errors or miscon- 
ceptions and suggestions for improvement that 
may prevent students from becoming discouraged 
(Ames, 1992; Bangert-Drowns, Kulik, Kulik, & Mor- 
gan, 1991; Boston, 2002; Elawar & Corno, 1985; Vis- 
poel & Austin, 1995). Students have the opportunity 
to acknowledge their limitations through candid 
self-appraisal without punitive consequences. This 
process facilitates the development of metacognitive 
skills that foster self-evaluation, enabling students 
at almost any level to develop a tool box of strategies 
for achieving target goals and transferring what is 
learned to new contexts (Assessment Reform Group, 
2002; Shepard et al., 2005). 



Promoting effective instructional practices 
across all districts within a state 



By adapting instructional strategies, formative 
assessment processes can be used to transform test 
results into meaningful learning activities (Fuchs 
et al„ 1997; Shepard, 2000a; Shepard et al„ 2005). 
They encourage teachers to develop instructional 
practices supported by research in cognitive 
science, measurement 
theory, and instructional 
technology (Chudowsky 
& Pellegrino, 2003; Stig- 
gins, 2002) and by the 
principles endorsed by 
the Commission on In- 
structionally Supportive 
Assessment (2001). 



By adapting instructional 
strategies, formative 
assessment processes 
can be used to 
transform test results 
into meaningful 
learning activities 



In addition, depending on the context, formative 
assessments could serve different instructional 
purposes, including: 

• Describing a student’s academic strengths and 
limitations. 

• Estimating a group’s knowledge level prior to 
instruction. 
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• Checking for misconceptions following 
instruction. 

• Providing evidence of progress toward learn- 
ing goals at key intervals during instruction. 

• Differentiating instruction. 

• Catalyzing curricular reform. 



accountability reform, see Goodwin, Englert, & 
Cicchinelli, 2003) suggest that access to key struc- 
tures and resources is likely to better position state 
decisionmakers to weigh the tradeoffs of each for- 
mative assessment option in relation to state goals 
for instruction. Specifically, targeted support for 
the following stakeholder groups has been shown 
to enable decisionmaking about instructional 
improvement through formative assessment: 



• Evaluating the effectiveness of an instruc- 
tional strategy. 

Educators have used results from formative assess- 
ments to measure, detect, diagnose, grade, pre- 
scribe, treat, observe, identify, and correct. Forma- 
tive assessments can be administered at different 
times in the instructional cycle (collected continu- 
ally, periodically, or at key intervals throughout 
a course of instruction), using different modes of 
administration (paper-based, computer-based, or 
performance-based) or formats (multiple-choice, 
open-ended, computer- adaptive, self-evaluation, 
journal, portfolio, or project; Commission on 
Instructionally Supportive Assessment, 2001; 
National Research Council, 2001a). 



Formative assessment 



processes can link 
the elements of a 
cohesive assessment 
system, including 
instruction, content 
standards, aligned 
curriculum, and trusted 
measures of progress 



Because formative assessments can serve differ- 
ent roles in the states, different models compete 
for educators’ attention (Black and Wiliam, 1998, 
2006; Olson, 2006; Shepard et al., 2005). Tests 
described as formative include curriculum-based 
assessment (Council of Chief State School Offi- 
cers, 2001), instructionally supportive assessment 
(Commission on Instructionally Supportive As- 
sessment, 2001), instructionally sensitive assess- 
ment (Shepard, 2000b), assess- 
ment for learning (Stiggins, 2002), 
and diagnostic assessment. 



Targeted research to facilitate 
decisionmaking about the most ef- 
fective formative assessment prac- 
tices is lacking. But findings from 
research in related areas (such as 
school reform, see Darling-Ham- 
mond & McLaughlin, 1995; and 



• Teachers, who need professional growth op- 
portunities to ensure development of content 
knowledge and specialized pedagogical skills 
that have been linked to effective instructional 
practices (Elmore, 2002; Shepard et al., 2005; 
Stiggins, 2002). 

• District and school administrators, who need 
the support of data management systems that 
facilitate documentation of assessment pro- 
cesses and appropriate reporting and use of 
results (Eleritage, Lee, Chen, & LaTorre, 2005; 
Reeves, 2004). 

• State policymakers, who need empirical data 
to defend decisions about assessment policies, 
programs, and practices (Pellegrino, 2006; 
Wiliam, 2005, 2007). 

Attention to these types of support can help 
ensure, across all districts, that formative assess- 
ment practices are consistent with state goals for 
effective instruction. 



Providing balance within a state's 
comprehensive assessment system 

No Child Left Behind-related demands for 
state and local education accountability require 
comprehensive assessment systems whose tests 
serve various purposes. To meet these needs, state 
assessment systems can be configured in several 
ways— from separate tests for each purpose (both 
summative and formative) to curriculum-based 
hybrids administered as standardized large-scale 
assessments (such as benchmark or interim as- 
sessments) but supporting finely grained analyses 
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BOX 2 

Methods and data collection 

This study examines evidence about 
state- and district-level formative 
policies, programs, and practices 
in the Southwest Region. Data were 
collected through document analyses 
and interviews with state and district 
representatives (see appendixes B, D, 
and E). Using case-study methodol- 
ogy, researchers then compared how 



Arkansas, Louisiana, New Mexico, 
Oklahoma, and Texas define forma- 
tive assessment, regulate formative 
assessment practices, and support the 
development and implementation of 
these practices at the state and district 
levels. They also describe a sample of 
locally initiated formative assessment 
practices drawn from these states. 

The findings suggest substantial vari- 
ability in how the Southwest Region 



states define formative assessment 
and in how each state guides or 
supports the development and use 
of formative assessments at the state 
and district levels. Because the find- 
ings are based on a small sample of 
states linked only by membership in 
a regional educational laboratory — 
and on the practices of a small conve- 
nience sample of districts — caution 
is warranted in generalizing findings 
beyond the immediate state sample. 



of student learning outcomes. Formative assess- 
ment processes can link the elements of a cohesive 
assessment system, including instruction, content 
standards, aligned curriculum, and trusted mea- 
sures of progress (Boston, 2002). 

The need exists both for summative data from 
large-scale achievement tests administered at 
the end of a year of instruction and for formative 
data from more individualized, short-term tests 
that provide descriptive and diagnostic informa- 
tion about students’ strengths and limitations 
(which may be used to inform decisionmaking 
throughout the course of instruction). Results 
from summative assessments may be useful in as- 
signing standardized judgments about a student’s 
proficiency in relation to state-adopted content 
standards. But results from formative assessments 
have been shown to be more effective in guiding 
the course of instruction in ways that have a posi- 
tive impact on student learning, especially for low- 
performing students (Black & Wiliam, 1998, 2006; 
Bloom, 1984; Crooks, 1988; Fuchs et al„ 1997; 
Natriello, 1987; Nyquist, 2003; Wiliam, 2007). 

The study presents findings from the document 
analyses and interview processes (see box 2 and 
appendix B on methodology and data collection) 
organized by the four research questions that 
guided this study (see appendix C for a side-by- 
side display of the evidence collected for each 
Southwest Region state on each of the research 
questions): 



• How are Southwest Region states defining 
formative assessment? 

• What state policies or programs related to 
formative assessment are in place in the 
Southwest Region? 

• What types of support link state policies with 
district formative assessment practices? 

• What examples of district-initiated formative 
assessment practices can be identified in each 
Southwest Region state? 



HOW ARE SOUTHWEST REGION STATES 
DEFINING FORMATIVE ASSESSMENT? 

Definitions of formative assessment vary across 
the Southwest Region states. They are also com- 
municated to stakeholders in different ways. Defi- 
nitions are situated in a variety of sources, includ- 
ing a professional development guide (Arkansas), 
an assessment guide (Louisiana), and a consumer 
guide to help teachers select formative assessment 
materials (New Mexico). 

Although researchers found explicitly stated defi- 
nitions in Arkansas, Louisiana, and New Mexico, 
they noted slight differences in how each of these 
definitions relates the purpose of formative assess- 
ment to teaching and learning. Whereas the Ar- 
kansas Department of Education Rules Governing 
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Professional Development (Arkansas Department 
of Education, 2005, 5.05.03, p. 7) defines forma- 
tive assessment as “assessment for learning,” 
Louisiana’s Graduate Exit Exam Assessment Guide 
(Louisiana Department of Education, 2006a, p. 
A-2) defines it as the “the ongoing evaluation of 
student performance for the purpose of assessing 
student learning and planning instruction.” And 
the New Mexico Public Education Department Di- 
vision of Assessment and Accountability Consumer 
Guide to Formative Assessments defines formative 
assessments as “ongoing assessments in a class- 
room . . . used to improve instructional methods 
and provide feedback throughout the teaching and 
learning cycle” (New Mexico Public Education 
Department, 2006b, p. 1). Researchers could find 
no evidence of formal definitions for Oklahoma or 
Texas in any paper or web-based document. 



WHAT FORMATIVE ASSESSMENT 
POLICIES OR PROGRAMS ARE IN PLACE 
IN SOUTHWEST REGION STATES? 



Although the Southwest Region states have vari- 
ous assessment policies, all have some type of 
formal state directive that can affect formative 
assessment practices. Such directives include state 
legislation, state education agency policies, and 
policies on released test items. 



Although the Southwest 
Region states have 



State legislation 



various assessment 
policies, all have 
some type of formal 
state directive that 
can affect formative 
assessment practices 



Evidence of a law, code, or regula- 
tion explicitly addressing formative 
assessment was found in Arkansas 
(house bill and state act), Texas 
(state administrative code), New 
Mexico (state act), and Oklahoma 
(school law). Louisiana regulates 
formative assessment processes 
only by default through an administrative code 
that calls for a comprehensive assessment system. 



State laws in Arkansas (Arkansas General Assem- 
bly, 1999) and Texas (Student Success Initiative 
of 1999; Texas Education Code Section 28.006) 



require the inclusion of formative assessment 
data in developing personal education plans for 
students failing to demonstrate proficiency on 
state tests or at risk of retention or not graduat- 
ing. Oklahoma’s Administrative Code 210 (2003) 
supports the use of diagnostic reading test data in 
association with its Reading Sufficiency Act/Read- 
ing Proficiency Act. New Mexico supports pre- and 
post-instruction tests for kindergarteners through 
its Reading Excellence Act (2000). 

The Arkansas Department of Education’s Rules 
Governing the Arkansas Comprehensive Testing, 
Assessment, and Accountability Program (2006c, 
7.02.4, p. 14) requires formative assessment 
strategies used in student academic improvement 
plans to be “included and revised periodically.” 
Filed in February 2007, Arkansas Elouse Bill 2253 
(Arkansas Elouse of Representatives, 2007) called 
for funding for a two-year pilot using formative 
assessments statewide. Allocations for the pilot 
were to be $25 per student (personal communi- 
cation with Arkansas Department of Education 
representative, April 2007). This bill has since been 
withdrawn, pending further study. 



State education agency policies or initiatives 

All Southwest Region state education agencies 
include formative assessment in their policies and 
programs, if in different ways. 

Arkansas. The Arkansas Department of Educa- 
tion Rules Governing the Arkansas Comprehensive 
Testing, Assessment, and Accountability Program 
(Arkansas Department of Education, 2006, 7.02.4, 
p. 14) requires that formative assessment strategies 
be included in all students’ academic improvement 
plans and that such policies be “revised periodi- 
cally based on results from the formative assess- 
ments.” According to a representative (interview, 
2007 March), the department authorizes the use 
of criterion-referenced tests in developing indi- 
vidual academic improvement plans for students 
failing to demonstrate proficiency on statewide 
tests in reading, writing, and math. The depart- 
ment’s Rules Governing Professional Development 
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(Arkansas Department of Education, 2005, 3.05, 
p. 1) specifies that “learning teams ... are to de- 
velop common formative assessments.” 

In addition, in a report from the Technology 
in Education Task Force commissioned by the 
Arkansas Legislative Joint Committee on Educa- 
tional Facilities (Technology in Education Task 
Force, 2004), task force members voice support for 
the development of an online formative assess- 
ment system that allows educators, students, and 
parents to monitor student progress through the 
use of test items linked to state standards. The Ar- 
kansas Department of Education administers the 
Smart Start (Arkansas Department of Education, 
2006e) and Smart Step (Arkansas Department of 
Education, 2006f) programs to monitor student 
progress toward state performance standards. The 
department also endorses the Arkansas Target 
Testing Initiative (discussed below). 

Louisiana. Ongoing formative assessment is 
called for in the Louisiana Department of Educa- 
tion’s Louisiana Literacy Plan (Picard, 2006). The 
department also administers the Practice Assess- 
ment/Strengthen Skills (PASS) program, which 
allows teachers to develop electronic and paper- 
based practice tests in English and language arts, 
math, science, and social studies to help students 
prepare for statewide testing (Louisiana Depart- 
ment of Education, nd). The department sup- 
ports use of the PASS system because, following 
research-based recommendations, it incorporates 
an instructional feedback loop. 

In May 2007 the Louisiana Department of Educa- 
tion secured a grant to develop its Enhanced As- 
sessments of Grade-Level Expectations (EAGLE) 
in response to teachers’ requests for finely grained 
performance data (substrand level) that will allow 
them to make meaningful data-based instruc- 
tional decisions. Teachers will have access to an 
online pool of custom items aligned to Louisiana 
standards in English and language arts, math, 
science, and social studies. Linked instructional 
resources give teachers additional tools for indi- 
vidualizing instruction. 



Of the Southwest 
Region states, Louisiana 
is the only one with 
a state-endorsed, 
department-initiated, 
online formative 
assessment tool 



EAGLE can be used 
diagnostically prior to 
instruction to develop or 
modify lesson plans or to 
evaluate level of compe- 
tency at the conclusion 
of an instructional unit. 

Student self-assessment 
and goal setting are 

encouraged through guided feedback that helps 
students identify the source of error or type of 
mistake and develop new learning skills to reach 
their goals. According to conversations with Ron 
May, project director, and Fen Chou, psychometri- 
cian (2007, April), the Louisiana Department of 
Education views this program as an opportunity 
to incorporate current research on formative as- 
sessment with a state-supported tool (customized 
item bank) for measuring student progress. Of 
the Southwest Region states, Louisiana is the only 
one with a state-endorsed, department-initiated, 
online formative assessment tool. 



New Mexico. The New Mexico Public Education 
Department convened a task force consisting of 
department specialists and district representa- 
tives to develop a consumer guide for formative 
assessments. The state’s Consumer Guide provides 
evaluations of vendors’ tests that district-level de- 
cisionmakers can use in determining the tests’ ap- 
propriateness for formative purposes (New Mexico 
Public Education Department, 2006b). These 
educator-written rubrics evaluate the strengths 
and limitations of each product according to 15 
criteria that include cost, link to state content 
standards, accessibility for students with disabili- 
ties and English language learners, and flexibility 
of administration. 



Vendors are encouraged to submit existing tests, 
and districts can request that certain products 
or programs be reviewed. The Consumer Guide 
is viewed as an interim measure to help districts 
evaluate existing tests until a statewide formative 
assessment is developed. The New Mexico Public 
Education Department web site encourages the 
use of the Consumer Guide during decisionmaking 
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to ensure that New Mexico districts have access 
to trustworthy information when judging forma- 
tive assessment systems. New Mexico is the only 
Southwest Region state that provides this type of 
statewide service for districts seeking to imple- 
ment formative assessment practices. 



According to Dr. Karen Harvey, assistant secre- 
tary, Quality Assurance and Systems Integration 
Division, (interview, 2007, April), the Public Edu- 
cation Department issued a request for proposals 
to investigate developing a voluntary state cur- 
riculum in math (2007/08), reading, and science 
(both 2008/09) with lesson plans and curriculums 
aligned to research-based guidelines and National 
Assessment of Educational Progress standards. 
When fully developed, this initiative is intended to 
support local formative assessments. 



All five Southwest Region 
states link their policies 
and programs to some 
form of state support for 
the development and use 
of formative assessments 



Oklahoma. According to guide- 
lines on its Student Assessment 
Division web site (Oklahoma State 
Department of Education, nd-c), 
the Oklahoma State Department of 
Education supports the use of data 
from the state’s exams for diag- 
nostic and formative purposes. 

Test score reports may be used diagnostically to 
measure student strengths and limitations and 
formatively to guide instruction and remediation. 



Texas. The Texas Education Agency supports 
formative assessment through the Student Suc- 
cess Initiative (Texas Education Agency 2007b), a 
broad statewide education initiative for students 
in grades 3, 5, and 8 who are at risk of retention 
because they did not pass the statewide tests in 
reading and math required for promotion at those 
grades. As one component of this initiative, grade- 
placement committees are expected to use forma- 
tive assessment data to develop personal education 
plans for these students. 



In addition, the Texas Education Agency monitors 
the Technology Immersion Project (Texas Educa- 
tion Agency, 2006b), which includes a platform 
for the administration of online diagnostic 



assessments. The Technology Immersion Proj- 
ect platform can deliver on-demand data about 
student proficiency so teachers can individual- 
ize instruction and implement interventions for 
individuals or student groups. Finally, using the 
Texas Primary Reading Inventory, Texas Educa- 
tion Agency staff (Anita Givens, director for in- 
structional materials and educational technology) 
administered a successful formative assessment 
pilot using hand-held devices for assessing early 
reading skills. 



Use of released test items 

All five Southwest Region states make released 
state test items available to the public through web 
sites maintained by state education agencies. In 
Oklahoma and Texas the use of released test items 
for formative purposes was codified in admin- 
istrative regulations. In Arkansas, Oklahoma, 
and Texas the state education agencies encourage 
stakeholders to access the released items specifi- 
cally for use in developing classroom-based forma- 
tive assessments. In Texas online practice tests for 
end-of-course exams also are available. Similarly, 
in Louisiana stakeholders have access to down- 
loadable state test items for practice in preparing 
for the statewide testing. 



WHAT TYPES OF SUPPORT LINK STATE 
POLICIES WITH DISTRICT FORMATIVE 
ASSESSMENT PRACTICES? 

All five Southwest Region states link their policies 
and programs to some form of state support for 
the development and use of formative assessments. 
These include professional development opportu- 
nities, resources, and product endorsements. 



Professional development opportunities 

In the Rules Governing Professional Development 
the Arkansas Department of Education (2005, p. 7) 
grants professional development time for “devel- 
oping assessments for learning.” It lists formative 
assessment as an approved topic for professional 
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development so that teachers can learn to develop, 
interpret, and use formative assessments in each 
subject area and develop common formative 
assessments. In addition, through its Education 
Service Cooperatives, the department supports 
staff development in using data from the Arkansas 
Target Testing Initiative (see below). The depart- 
ment also supports training for K-3 and special 
education teachers seeking to use reading program 
data formatively in developing individual inter- 
vention plans. Finally, it monitors federal funds 
associated with the Positive Behavior Support 
Project, which provides grant money to educators 
for training in using discipline data formatively as 
an intervention for students with school-behavior 
issues (James, 2004). 

In all states except New Mexico the state education 
agencies support professional development sessions 
related to formative assessment delivered by a test 
vendor, Educational Testing Service (ETS; Educa- 
tional Testing Service, 2007), which offers profes- 
sional development to administrators and teachers 
from participating districts through “Webinars.” 
The web-based interactive sessions assist teachers 
in using the ETS Formative Assessment Item Bank 
as a tool. The Texas Technology Immersion Pilot 
(see above) includes a professional development 
component in addition to technical support. Loui- 
siana’s EAGLE includes a professional development 
component, as does its PASS (see above). 



Resources 

Through the Louisiana Department of Educa- 
tion, the American Education Corporation offers 
students displaced by Hurricane Katrina free 
access to online assessments so that schools have 
a diagnostic tool for judging skill levels in math, 
reading, science, and social studies. New Mexico’s 
Consumer Guide (New Mexico Public Education 
Department, 2006b) provides support to local 
education agencies with informational resources 
that can be used in selecting formative assessment 
products. Through a collaborative project with the 
University of Arkansas, the Arkansas Department 
of Education supports state teachers through its 



Enterprise Guide for Educators, which guides them 
in accessing data for use in the improvement of 
classroom instruction (National Office for Re- 
search, Measurement, and Evaluation, 2007). 

Texas’ Student Success Initiative (see above) 
includes support for district use of Pearson’s 
Progress Assessment Series (Pearson, 2007b), an 
online system for diagnosing student strengths 
and limitations in grades 3, 5, and 8. The Texas 
Education Agency also provides technical support 
to district staff through its Technology Immersion 
Project (Texas Education Agency, 2006b). 



Product endorsements 

The Arkansas Department of Education endorses 
the Triand web-based system (Arkansas Depart- 
ment of Education, nd) for managing data associ- 
ated with formative testing at the district level. 

The Oklahoma State Department of Education 
supports use of a formative assessment resource 
called Data Connections (Appalachia Education 
Laboratory and Edvantia, 2003). The Texas Educa- 
tion Agency, through its Student Success Initiative, 
encourages districts to use Pearson’s Progress As- 
sessment Series (Pearson, 2007b). 



WHAT EXAMPLES OF DISTRICT-INITIATED 
FORMATIVE ASSESSMENT PRACTICES CAN BE 
IDENTIFIED IN EACH SOUTHWEST REGION STATE? 



Through web searches, document analyses, and 
interviews with state department of education staff 
17 district and multidistrict initiatives were identi- 
fied. The local education 
agencies have imple- 
mented these formative 
assessment processes 
using custom-developed 
or off-the-shelf formative 
assessments. Districts 
in Arkansas, Louisiana, 

Oklahoma, and Texas 
have adopted the ETS 
Formative Assessment 



The study identified 17 
district and multidistrict 
initiatives. The local 
education agencies have 
implemented these 
formative assessment 
processes using custom- 
developed or off-the-shelf 
formative assessments 
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Item Bank (Educational Testing Service, 2007) 
for the development of online assessments. Those 
in Arkansas and Louisiana use Learning Today’s 
Smart Tutor web-based system (Learning Today, 
2006) to inform differentiated instruction. Those 
in New Mexico and Texas use ThinkLink Predic- 
tive Learning Series (Discovery Education, nd) for 
periodic, benchmark-based online testing. Those 
in Oklahoma use Scantron’s diagnostic programs 
(Scantron Corporation, 2007). 

One district-level or multidistrict level formative 
assessment initiative from each Southwest Region 
state was selected for highlighting in this report. 
These initiatives do not represent best practices. 
Instead, each was derived from a convenience 
sampling process and is included only to add a 
variety of local perspectives to the findings. 



Arkansas 



The Arkansas Target 
Testing Initiative, 
a unique Arkansas 
Department of 
Education-endorsed 
formative assessment 
program, provides 
participating districts 
with a cost-effective 
formative assessment 
tool. Math and literacy 
items have been 
developed by content 
experts in alignment 
with state learning 
expectations 



The Arkansas Target Testing Initiative (South 
Central Service Cooperative, nd), a multidistrict 
initiative developed by districts through their Edu- 
cation Service Cooperative, is unique. An Arkan- 
sas Department of Education-endorsed formative 
assessment program, it provides participating 
districts with a cost-effective formative assessment 
tool. Math and literacy items have been developed 
by content experts in alignment 
with state learning expectations. 



Multiple- choice and constructed- 
response items in the databank 
are used to diagnose students’ 
strengths and limitations. Re- 
sponses to multiple- choice items 
are scanned and scored mechani- 
cally, with results available to 
students almost immediately. Re- 
sponses to constructed-response 
items are scored by district teams, 
with a slightly longer turn-around 
time, for each content area teach- 
ers are provided with instructional 
pacing guides to focus on student 
strengths and weaknesses prior 



to the state exams. Workshops for participating 
districts facilitate the effective use of data. 



Louisiana 

To monitor student learning, Union Parish 
School District in Larmersville uses Pearson’s 
Classroom Performance System (Pearson, 2007a). 
Using hand-held devices, students assess their 
own learning processes during an instructional 
segment and communicate learning outcomes to 
the teacher. Teachers can track comprehension, 
administer self-paced testing, and create custom 
student and class reports in a variety of formats 
for different audiences. District staff have found 
the tool useful for keeping students on-task and 
raising their performance levels. 



New Mexico 

The Las Cruces Public School District includes 21 
schools that did not reach the state’s annual No 
Child Left Behind performance goals. To sup- 
port educators in reaching performance goals for 
all students in 2007/08, the district has decided 
to implement Measures of Academic Progress 
(MAP), developed by the Northwest Evaluation 
Association (2003). Using the state’s Consumer 
Guide, MAP was identified by district staff as the 
formative assessment process that would best meet 
their needs and goals. 

MAP is expected to provide district teachers with 
timely, accurate performance data that will enable 
them to focus on each student’s unique instruc- 
tional needs. The district is planning forums for 
principals and teachers to discuss implementa- 
tion issues, with topics ranging from the efficacy 
of the process to concerns about technology 
infrastructure. 

In addition, the Las Cruces Public Schools Dis- 
trict supports student achievement by explicitly 
incorporating formative assessment processes 
in its Curriculum Alignment Matrix (Las Cruces 
Public Schools, 2006). This print and online tool 
for teachers combines content-specific pacing 
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guides with state and classroom test results and 
guides teachers in using student assessment data 
formatively. 



Oklahoma 

The Jenks Public Schools District uses two pro- 
grams for formative assessment purposes. The first 
is a diagnostic system in which pre- and post-tests 
are administered prior to and following instruc- 
tion. Test items for these tests have been developed 
in math, science, and writing by district teachers; 
items for the social sciences are scheduled for de- 
velopment in 2007/08. District staff review the pre- 
and post-tests on an ongoing basis to ensure that 
they remain reliable measures of performance and 
that valid inferences are drawn from findings. This 
process is intended to provide Jenks Public Schools 
teachers with valuable professional development 
in item writing and with diagnostic information 
about student strengths and limitations. 

The second program used by Jenks Public Schools 
is the Essential Elements program (Jenks Public 
Schools, 2007), which is focused on developing 
essential elements, or instructional priorities for 
each class. Ongoing testing on those elements 
tracks student growth over time. Graphs depicting 
individual and group growth are posted for review 
by class members, with results used formatively 
to inform instructional decisionmaking. Jenks 
Public Schools staff are provided with professional 
development opportunities associated with the 
Essential Elements program. 



Texas 



The Clarksville Independent School District is a 
rural district that faces challenges associated with 
higher-than-state-average rates of poverty and 
unemployment. It received a Texas Technology 
Immersion Program (Texas Education Agency, 
2006b) grant to focus on raising student achieve- 
ment through technology-based formative assess- 
ments. Students and teachers in three schools in 
grades K-12 are provided with notebook com- 
puters for administering online tests linked to 



key learning outcomes. 
Teachers have access to 
professional development 
workshops in using elec- 
tronic data formatively 
to improve instructional 
methods and promote 
student learning. 



The Clarksville 
Independent School 
District received a Texas 
Technology Immersion 
Program grant to focus 
on raising student 
achievement through 
technology-based 
formative assessments. 
Students and teachers 



are provided with 
notebook computers 
for administering online 
tests linked to key 
learning outcomes 



The Technology Immer- 
sion Program (Texas 
Education Agency, 2006b) 
is used as the platform 
for administering CTB/ 

McGraw-Hill’s i-Know, 
an online process for 
measuring student skills 

and tracking student progress (CTB/McGraw-Hill, 
2007). I-Know includes items developed from both 
national and Texas content standards. Through 
ongoing online testing, teachers can assess student 
skills in reading, language, applied math, math 
comprehension, science, and social studies. 

District teachers using this system are provided 
with immediate feedback on student strengths and 
limitations, measured while students are engaged 
in learning activities. 



CALL FOR RESEARCH IN THE FIELD 

Recently, state policymakers have joined educa- 
tors in considering the significant role of forma- 
tive assessment in providing students with the 
targeted support they need to achieve grade-level 
expectations. But because of the dearth of empiri- 
cal evidence on the impact of state policies on 
formative assessment practice, more research is 
needed to inform state decisionmakers seeking to 
understand how state-level guidance — whether 
through policy, programming, or resource 
allocation — may foster effective state and local 
formative assessment practices and ensure cross- 
district consistency with state education goals. 
Clearer direction would better position state 
education agencies to evaluate formative assess- 
ment policy options in light of tradeoffs related to 
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state- and district-specific needs 
and goals. 

Two sets of findings from this 
study are intended to serve as a 
catalyst for future research. First, 
among Southwest Region states, 
even states facing similar chal- 
lenges (box 1 table) — in the size 
of the K-12 student population 
(Arkansas and New Mexico, or 
Louisiana and Oklahoma), the percentages of 
students with disabilities (Arkansas and Texas), or 
the percentages of English language learners (New 
Mexico and Texas) — have chosen different paths 
for developing or supporting formative assessment 
at the state and local levels. What student needs 
precipitate movement toward a specific type of 
state-supported formative assessment initiative? 
What factors or conditions related to the student 
population may influence a state’s decision to 
regulate formative assessment practices? It would 
be useful for researchers to develop working 



Clearer direction would 
better position state 
education agencies 
to evaluate formative 
assessment policy 
options in light of 
tradeoffs related to 
state- and district- 
specific needs and goals 



hypotheses about the nature of these relationships 
and explore them empirically. 

Second, in the small sample of districts within the 
Southwest Region (N=17) a range of district-level 
formative assessment practices have emerged. 
These include use of online test item banks (except 
New Mexico), noncustomized products (all states), 
diagnostic tools (Arkansas, Louisiana, Oklahoma), 
end-of-course or benchmark assessments (New 
Mexico, Oklahoma), district-developed custom- 
ized programs (New Mexico), and collaborative 
cross-district efforts (Arkansas). What district 
characteristics need to be considered when de- 
veloping a formative assessment initiative? What 
types of initiatives are most effective in address- 
ing specific district needs? What types of state 
support sustain formative assessment initiatives? 
Additional research is warranted to examine more 
systematically all formative assessment practices 
within each state and to explore the extent to 
which the existing or emerging initiatives are 
enabled by state guidance or support. 
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APPENDIX A 
STATE STATISTICS 



TABLE A1 

State summary statistics 



1 Statistic 




Arkansas 


Louisiana 


New Mexico 


Oklahoma 


Texas 


K— 1 2 student enrollment 




463,115 


724,281 


326,102 


629,476 


4,405,215 


Number of public districts 




318 


68 


89 


541 


1,039 


Number of public schools 




1,158 


1,541 


842 


1,787 


8,746 


Elementary schools 




600 


831 


449 


1,020 


4,224 


Middle schools 




201 


245 


152 


294 


1,576 


High schools 




311 


303 


119 


483 


1,687 


Multilevel schools 




— 


156 


— 


— 


469 


Alternative schools 




5 


36 


27 


— 


714 


Career/technical schools 




— 


— 


— 


54 


— 


Charter schools 




17 


17 


44 


12 


321 


Student characteristic (percent) 


Asian 




1 


1 


1 


2 


3 


Black 




23 


48 


3 


11 


14 


Hispanic 




6 


2 


53 


8 


45 


American Indian 




1 


1 


11 


19 


<1 


White 




69 


48 


32 


61 


38 


Eligible for free or reduced-price lunch (percent) 


52 


62 


58 


54 


48 


Students with disabilities (percent) 




12 


14 


20 


15 


12 


English language learners (percent) 




4 


2 


19 


7 


16 


Graduation rate (percent) 




81 


83 


84 


86 


84 


Mean percent at or above proficient 
2006 state English language 


Grades 3-5 


50 


64 


55 


77 


81 


Grades 6-8 


55 


52 


47 


73 


83 


arts and reading 


Mean percent at or above proficient 
2006 state math 


Grades 3-5 


50 


61 


36 


74 


81 


Grades 6-8 


40 


54 


22 


69 


66 


Mean scale score 

2005 state National Assessment 


Grade 4 


217 


230 


224 


214 


219 


Grade 8 


258 


268 


263 


260 


258 


of Educational Progress reading 


Mean scale score 

2005 state National Assessment 


Grade 4 


236 


209 


207 


234 


242 


Grade 8 


272 


253 


251 


271 


281 


of Educational Progress math 



— is not available. 

Source: U.S. Department of Education, 2006; National Office for Research, Measurement, and Evaluation, 2005 (Arkansas); Louisiana Department of Educa- 
tion, 2005 (Louisiana); New Mexico Public Education Department, 2005; Oklahoma State Department of Education, 2005 (Oklahoma); Texas Education 
Agency, 2005 (Texas). 
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APPENDIX B 

STUDY METHODS AND LIMITATIONS 

This study used case study research to investigate 
a contemporary phenomenon within its real-life 
context (Yin, 1984). Case study research drawn 
from recent policy analyses (McDermott, 2007) is 
appropriate when researchers seek to build on what 
is already known by collecting evidence to identify 
common patterns while shaping a detailed portrait. 
The case study researcher relies on systematic, in- 
depth contextual analyses and the triangulation of 
data to strengthen research findings and conclu- 
sions (Glaser & Strauss, 1967; Stake, 1995). 

Case study methods offer specific advantages for 
this study. When implemented appropriately, they 
help ensure reliability of the processes and valid- 
ity of inferences drawn from findings (Merriam, 
1998). These advantages include: 

• Research design well suited to creating a 
descriptive portrait of each state. 

• Pre-data collection planning about key 
sources and types of evidence for addressing 
the research questions. 

• Multiple data sources and data-gathering 
techniques to build a chain of evidence about 
formative assessment. 

• Careful documentation of sources of evidence. 

• Established protocols for interview processes. 

• Appropriate reporting format. 



Planning and pre-data collection steps 

The state was identified as the unit of analysis 
(N = 5). Following case study guidelines, plans 
were initiated for the systematic collection, catego- 
rization, storing, and reporting of state-specific 
evidence. Researchers were trained to identify 
and record specific types of descriptive data, to 
be persistent in locating elusive information, and 



to remain open to conflicting data and contrary 
findings. 

Side-by-side tables were created (appendix C) to 
display data for each Southwest Region state as 
they were collected. This type of data organizer 
was intended to facilitate reporting of the ways 
formative assessment was being defined, sup- 
ported, and implemented across study states. Each 
category of evidence in the table was linked spe- 
cifically to one or more of the research questions. 

This table was designed to be dynamic so adjust- 
ments or modifications could be made, as needed, 
during the data collection phases. For example, in 
two cases emerging data suggested that a category 
might more clearly be addressed by dividing it into 
subcategories. This strategy was applied to research 
question 2 (table C3 in appendix C), where state 
policies or programs related to formative assess- 
ment were further subdivided into state regula- 
tions, codes, and laws; state education agency poli- 
cies or programs; and policies related to released 
test items — and to research question 3 (table C4 in 
appendix C), where types of support were further 
subdivided into professional development opportu- 
nities, resources, and product endorsements. 

During early planning sessions researchers also 
identified state characteristics, related to the 
K-12 student population served, that might vary 
systematically with differences in formative as- 
sessment policies, programs, and practices. Such 
characteristics have been shown to be associated 
with cross-state differences in test performance on 
the National Assessment of Educational Progress 
(U.S. Department of Education, 2007; Standard 
& Poor’s, 2005). Therefore, data describing these 
characteristics for each Southwest Region state 
were collected to provide context for the data col- 
lected through document analyses and interview 
processes. Providing this type of background 
information about the unit of analysis is recom- 
mended in case studies (Merriam, 1998). 

Finally, a protocol for the telephone interviews 
with state and district staff was developed. These 
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interviews were intended to verify, update, and 
supplement data collected through the document 
analyses. Researchers anticipated challenges 
in accomplishing this goal. Researchers under- 
stood that it might be difficult to locate the most 
knowledgeable representative about formative 
assessment within a state department of educa- 
tion, because that staff person could be assigned 
to several divisions within that department, such 
as assessment, accountability, curriculum and 
instruction, or special populations. Locating the 
most knowledgeable district representative was 
also likely to prove challenging because the target 
staff person could be assigned to a school rather 
than the district office. As a result, researchers 
concluded that an informal interview structure — 
one that accommodated impromptu interviews, 
if needed — would be most appropriate, using 
prompting questions only when needed. Interview 
time was estimated at 30 minutes. 



Review of literature 

A focused literature review was conducted to serve 
as the foundation for the research questions and to 
provide additional context for the descriptive data 
collected in the document analyses. During this 
process, the researchers’ scope of review was not 
restricted to information about Southwest Region 
states. Instead, their review included policies and 
practices related to formative assessment that 
extend across all states. 

Initially, the literature review focused on locat- 
ing nationally disseminated empirical research 
on the purposes of formative assessment and the 
effects of state policy on state and local formative 
assessment practice. Data about the purposes of 
formative assessment were accessible. But despite 
searches across a range of sources that included 
professional journals, assessment-related web sites, 
national databases, and proceedings or programs 
from professional conferences (see table Bl), no 
study was located specifically describing the 
effects of state formative assessment policies on 
local practice. As a result, researchers expanded 
their review to include studies of the effects of 



state policy in related areas, such as school reform, 
summative assessment, and accountability. 

Becaus e formative assessment refers to a genre of 
assessment processes and different formative as- 
sessment models compete within the education re- 
search community (Black and Wiliam, 1998, 2006; 
Olson, 2006; Shepard et al., 2005), researchers 
relied on a range of key words to access relevant 
information in research reports and to locate links 
to empirical research studies, formal presenta- 
tions, and policy reports. The key words included: 

• Formative, diagnostic, benchmark, interim, 
curriculum-based, and instructionally-support- 
ive or instructionally-sensitive assessments. 

• Pre-post testing. 

• Assessment for learning. 

• Guidelines for data use. 

• Instructional or curricular adaptations and 
differentiated instruction. 

• Metacognitive strategies, cognitive scaffold- 
ing, and the feedback cycle. 

• Released test items. 

• Comprehensive assessment system. 

• Assessment-related state policies and programs. 

These terms may have been associated with a state 
or district policy or program; written communi- 
cation between the state and school districts; a 
state or district assessment, test, process, tool, or 
instrument; or a state or district data system. 



Data sources, data collection steps, and 
documentation of evidence 

Data collection efforts during this phase of the 
study were focused on information specifically 
related to the Southwest Region states. In keeping 
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TABLE B1 

Summary of data collection activities 



1 Resource type 


Example or explanation 


Documents analyzed 


State Department of 
Education or other state- 
maintained web sites 


Web pages for State Report Card or state assessment 
programs 


State codes, regulations, policies, 
publications, communications, 
presentations, reports, annual report 
cards, technical manuals, interpretive 
guides, Peer Review Workbook 


U.S. Department of 
Education or other 
federally maintained web 
sites 


National Center for Education Statistics or National 
Assessment of Educational Progress web pages 


Peer Review letters, state-by-state 
demographic and performance 
summary data 


National organizations or 
annual conferences 


Council of Chief State School Officers, Education 
Commission of the States, Phi Delta Kappa, American 
Educational Research Association, National Council 
on Measurement in Education, National Center for 
Educational Accountability, National Institute of Standards 
and Technology 


Web pages, articles, reports, 
presentations, proceedings, 
programs, announcements, state-by- 
state comparisons 


National databases or 
clearinghouses related to 
assessment 


Assessment and Accountability Comprehensive Center, 
Center for Research in Evaluation, Standards, and Student 
Testing 


Vetted research articles, position 
papers, policy statements 


State university web sites 


University of Arkansas 


Articles, reports, presentations 


Test vendors or 
organizations involved 
with testing 


Educational Testing Service, Pearson, ThinkLink, Scantron 
Corporation, Learning Today, Princeton Review, Northwest 
Evaluation Association 


Web pages, pamphlets, brochures, 
reports, announcements, press 
releases, sample score reports 


School district web sites 


Clarksville Independent School District, Texas; Jenks 
Public Schools, Oklahoma; Las Cruces Public Schools, New 
Mexico; Little Rock School District, Arkansas; Union Parish 
School Board, Louisiana 


Web pages, announcements, reports, 
annual report cards 


Books, newspapers, and 
research journals 


American Educational Research Journal; Education 
Week; Educational Leadership; Educational Psychologist; 
Journal of Educational Psychology; Journal of Educational 
Research; Kappan; Practical Assessment, Research, and 
Evaluation; Review of Educational Research 


Articles, chapters, special editions 


Telephone interviews 


One per state (N = 5), with state-level representative 
identified as knowledgeable about the state's formative 
assessment initiatives 


Oral verification of state policies/ 
programs, possible links between 
state policies and programs and 
district initiatives 


Telephone interviews and 
email exchanges 


One per state (N = 5), with district-level representative 
identified as knowledgeable about a district formative 
assessment initiative 


Oral verification of district practice, 
possible links between state policies 
and programs and district initiatives 



Source: Authors' analysis based on data described in text. 



with recommendations for case study methodol- 
ogy (Merriam, 1998), multiple data sources and 
data collection strategies were used to allow the 
triangulation of findings on Southwest Region 
state formative assessment policies, programs, and 
practices. These sources included paper-based and 
web-based documents as well as interviews with 
state and district representatives. Data collection 



activities extended from September 2006 through 
August 2007. 



Analysis of documents and interviews with administrators 

Because analysts were focused on answering the 
study’s research questions, they sought informa- 
tion about how Southwest Region states defined 
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formative assessment and operationalized it in 
communication with districts, state policies or 
programs related to formative assessment, state 
guidelines for formative data use, forms of state 
support for formative assessment, and types of 
formative assessment practices at the district level. 
As a first step in this process, analysts identified 
sources for such information. (Table B1 summa- 
rizes data sources by resource type.) 

Analysts carefully reviewed documents for 
evidence related to the four research questions. 
Documents that included a reference to a topic of 
interest were recorded, the information cataloged, 
and relevant data added to the side-by-side table 
(table Cl). Through an iterative process, the table 
was continually updated as new information was 
located, so that a comprehensive descriptive por- 
trait of the status of formative assessment policies 
and practices in each of the five Southwest Region 
states emerged. 

When gaps in coverage across the states were 
noted — such as a policy definition of formative as- 
sessment for one state — analysts persistently and 
methodically pursued alternate sources of infor- 
mation. They attempted to exhaust all possible 
information sources, including those available to 
the public and those restricted to members (such 
as the American Educational Research Asso- 
ciation) to ensure that the data presented in the 
side-by-side table were both exclusive and compre- 
hensive. They intended to verify these assurances 
during the interviews with state personnel, who 
may have had access to restricted or proprietary 
information. 

One source of information about operationaliza- 
tion of formative assessment policy was state 
policy on released items. To ensure that stakehold- 
ers have access to representative samples of the 
types of knowledge and skills assessed by their 
states’ annual high stakes, NCLB accountability- 
based tests of achievement, states release test items 
or test forms each year for public review. This 
practice is also endorsed by the National Center 
for Education Statistics for National Assessment 



of Educational Progress assessments (U.S. Depart- 
ment of Education, 2007) and the International 
Association for the Evaluation of Educational 
Achievement for Trends in International Math and 
Science Study assessments (2005). Because states 
may promote formative use of these “released” 
items, data were collected related to policies on the 
use of released state test items. 

Interviews with department of education represen- 
tatives. To orally verify, update, and supplement 
data collected through the document analyses, 
researchers arranged for telephone interviews with 
a state department of education staff person from 
each of the five states. Before scheduling the inter- 
views, the Southwest Regional Education Labora- 
tory at Edvance Research Institute sent a letter to 
the chief school officer for each state to formally 
introduce the formative assessment study, provide 
details about its purpose and research compo- 
nents, and inform the chief of the intent to contact 
a staff member from the state’s department of 
education. 

Researchers then contacted the assessment divi- 
sion of each state’s department of education for 
a referral to the one representative considered 
most knowledgeable about formative assessment 
policies and programs in that state. As anticipated, 
across states, formative assessment responsibili- 
ties (if existing) currently are assigned to staff in 
several divisions, including assessment, account- 
ability, curriculum and instruction, special popu- 
lations, and district outreach. 

As a result, identifying the most knowledgeable 
representative was a multistep process, as those 
originally identified often referred researchers to 
other staff members once the specific nature of 
the information needed was clarified. Over time, 
though some information may have been gathered 
from each of these contacts, one key staff member 
from each state was identified as most appropriate 
for a more thorough interview. In three of the five 
states this chain of referrals led to the most appro- 
priate staff member in an impromptu manner, so 
researchers did not provide written summaries of 
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collected data prior to the interview. In the other 
two states written summaries were emailed to the 
staff member prior to the scheduled interview. 

The structure for these interviews was informal, 
with each interview lasting 15-30 minutes. A 
trained researcher introduced the study, explained 
its purpose, and sought to foster an atmosphere of 
collaboration and trust. In all cases the researcher 
summarized existing information orally. The staff 
member was asked to consider the collected data, 
to offer clarifying comments when relevant, and 
to provide oral confirmation of the accuracy of 
the data, when possible. The representative was 
encouraged to share additional information about 
the state’s formative assessment policies and pro- 
grams related to the research questions. To elicit 
key information or to fill gaps that had emerged 
in data collected for that state, researchers used 
prompting questions, such as: 

• Has the status of any of these data changed 
recently? 

• Is the state considering implementing any 
new policies or programs? 

• Does the department monitor district or 
school formative assessment initiatives? 

At the conclusion of each telephone interview the 
researcher asked the department representative 
to recommend a district in the state currently 
engaged in a formative assessment initiative. In all 
but one state interviewees stated that they did not 
have reliable information related to district prac- 
tices and could not offer a recommendation. Data 
were collected from the researcher’s hand-written 
notes and added to the side-by-side table, where 
appropriate. 

Interviews with district representatives. To learn 
about how state policies and programs might 
be operationalized and to provide information 
about the types of formative assessment initia- 
tives developed at the district level, researchers 
identified a pool of districts in the five Southwest 



Region states engaged in formative assessment 
practices. For this purpose, using the document 
analysis strategies described above, they collected 
information about district formative assessment 
activities. Summarized in table Bl, data about 
district formative assessment practices were 
sought from national and Southwest Region state 
web sites; research journals, organizations, and 
clearinghouses; professional conferences; research 
or policy reports; press releases; and test vendor 
publications. 

Using these data, researchers identified 17 unique 
districts or multidistrict consortia across the five 
states as potential contacts for district interviews. 
All were districts with formative assessment 
practices recognized on state or vendor web sites, 
highlighted in a local newspaper or professional 
journal, or awarded recognition through national, 
state, or local announcements. As the one district 
name provided by the state-level representative was 
a district already identified through document anal- 
yses, these 17 districts represented the final pool 
for drawing the convenience sample of five districts 
(one per state). The final list of districts included: 

• Arkansas: Little Rock School District, Ash- 
down Public Schools, Boonville School Dis- 
trict, El Dorado School District. 

• Louisiana: Union Parish School District, Jef- 
ferson Parish School District. 

• New Mexico: Las Cruces Public Schools, 
Espanola Public School District, Albuquerque 
Public Schools, Taos Municipal Schools, Los 
Lunas School District. 

• Oklahoma: Jenks Public Schools, Bristow 
Public Schools, Blackwell Public Schools. 

• Texas: Clarksville Independent School Dis- 
trict, Aldine Independent School District, 
Hildalgo Independent School District. 

From this pool five districts or multidistrict 
consortia (one from each Southwest Region state) 
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were selected that, based on existing intorma- Study limitations 



tion, represented a range of different formative 
assessment practices (item bank, noncustomized 
diagnostic test, customized process). Researchers 
sent letters of introduction to the superintendents 
of each district to explain the study’s purpose and 
ask for a referral to a representative knowledge- 
able about the district’s formative assessment 
initiative (appendix D). Responses were received 
from two of the five districts; telephone inter- 
views were scheduled with the recommended 
representative from each district. No response 
was received from the other three districts, so 
researchers followed up with telephone calls. This 
process resulted in one impromptu telephone 
interview. The remaining two districts agreed to 
email written responses to a set of questions (ap- 
pendix E). 

The structure for the two telephone interviews 
was informal, with each interview lasting 15-20 
minutes. A trained researcher introduced the 
study, explained its purpose, and sought to foster 
an atmosphere of collaboration and trust. The 
researcher summarized existing data about the 
district’s formative assessment initiative and 
asked the representative to share any additional 
information about this initiative. To prompt their 
responses, researchers asked such questions as the 
following: 

• What did your district hope to accomplish 
with this initiative? 

• How are formative assessment data currently 
used in your district? 

• What resources, financial or otherwise, were 
provided by the state to support 

your initiative? 

At the conclusion of each telephone interview data 
were collected from the researcher’s hand-written 
notes and added to the side-by-side table, where 
appropriate. Responses received by email (two dis- 
tricts) were also reviewed, and relevant data were 
included on the side-by-side table. 



As in other studies collecting evidence using the 
case study methodology, researchers faced special 
challenges ensuring the reliability of data collec- 
tion measures and the validity of findings (Mer- 
riam, 1998). Therefore, safeguards were imple- 
mented to ensure exhaustive and comprehensive 
searches for information about formative assess- 
ment in both print and electronic formats, careful 
documentation of data collection strategies, and 
sources of evidence to enable study replication. 

First, two researchers worked simultaneously over 
12 months (September 2006 through August 2007) 
to identify reliable sources of information and to 
verify findings. Using an iterative process, state 
data were updated weekly through ongoing but 
focused searches to confirm existing information 
and identify new print or electronic resources. Sec- 
ond, multiple methods facilitated the triangulation 
of emerging findings. Findings from document 
analyses were supported by data from telephone 
interviews with state education agency representa- 
tives that served as a check on data collected, as 
well as data from telephone interviews with local 
education agency representatives that offered 
an opportunity to supplement findings from the 
document analyses and state-level interviews with 
district-confirmed information. 

This investigation was conducted with diligence 
in seeking and exploring all sources of evidence. 
Researchers systematically followed case study 
recommendations, using dependable data col- 
lection methods appropriate for the purpose 
and giving principled attention to accuracy and 
validity in reporting findings. Nonetheless, these 
findings represent only a snapshot of state poli- 
cies, practices, and programs regarding formative 
assessment that were evident at a point in time. 
Additionally, the findings are based on a small 
sample of states that are linked only by member- 
ship in a regional educational laboratory — and 
on the practices of a small convenience sample of 
districts. So, caution is warranted in generalizing 
findings beyond the immediate state sample. 
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The results from this examination between state 
policy and district practices are not intended to 
imply causal relationships. In many cases, district 
practices were initiated before implementation of 
state policy, and district practices were unrelated 
to state initiatives. For this reason the authors 
avoid drawing inferences about the effectiveness of 
any policy, program, or practice. 
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APPENDIX C 

SIDE-BY-SIDE COMPARISON OF STATES' 

FORMATIVE ASSESSMENT POLICIES, 

PRACTICES, AND PROGRAMS 

TABLE Cl 

Information about comprehensive assessment systems 



Peer review status: 

State Information standards and assessments 



Arkansas 


State tests: Arkansas Comprehensive Testing and Accountability Program 
(ACTAAP) benchmark exams (criterion-referenced tests, grades 3-8); end-of-course 
exams (algebra, geometry, literacy); Iowa Tests of Basic Skills (ITBS; grade 3-8); 

Iowa Test of Education Development (ITED) at grade 9; and Arkansas Alternate 

Portfolio Assessment System (AAPAS) for students with significant cognitive Full approval 

disabilities (Arkansas Department of Education, 2006a, 2006b, 2006c, 2006d). (December 19, 2006) 


Louisiana 


Comprehensive assessment system: iLeap (norm-referenced test augmented 
criterion-referenced test in grades 3, 5-7, 9); Louisiana Educational 
Assessment Program (LEAP) (criterion-referenced test in grades 4 and 8); 

graduation exit exam (GEE); Louisiana alternate assessments (LAA1 and Approval expected 

LAA2) (Louisiana Department of Education, 2006a, 2007b, 2007c, 2007d). (June 29, 2006) 


New Mexico 


New Mexico Achievement Assessment Program (NMAAP): New Mexico 
Standards Based Assessment (NMSBA, grades 3-9 and 11); New Mexico 
High School Competency Exam (NMHSCE); New Mexico English Language 

Proficiency Assessment (NMELPA); New Mexico Alternate Performance Approval pending 

Assessment (NAPA) (New Mexico Public Education Department, nd). (June 22, 2006) 


Oklahoma 


Oklahoma School Testing Program (OSTP) based on core curriculum 
(Priority Academic Student Skills — PASS): Oklahoma Core Curriculum 

Tests (OCCT, grades 3-8), end-of-course tests (Algebra, Geometry, Full approval 

Literacy) (Oklahoma State Department of Education, nd-a, nd-c). (June 30, 2006) 


Texas 


Texas Assessment Program (TAP): Texas Assessment of Knowledge 
and Skills (TAKS and TAKS-ALT; grades 3-HS in English and language 
arts, math, science, social studies), the State Developed Alternative 
Assessment (SDAA II), the Texas English Language Proficiency Assessment 

System (TELPAS), and the Texas Assessment of Academic Skills (TAAS, Approval pending 

being phased out as exit exam) (Texas Education Agency, nd-a). (October 27, 2006) 



TABLE C2 

Research question 1: How are Southwest Region states defining formative assessment? 



State Defining formative assessment 



| | State 


Defining formative assessment 


Arkansas 


Assessments for learning (5.05.03.4, Arkansas Department of Education, 2005). 


Louisiana 


Formative assessment is the ongoing evaluation of student performance for assessing student learning and 
planning instruction (Louisiana Department of Education, 2006a). 


New Mexico 


Formative assessments are ongoing classroom assessments used to improve instructional methods and 
provide feedback throughout the teaching and learning cycle (New Mexico Public Education Department, 
2006). 


Oklahoma 


No evidence. 


Texas 


No evidence. 
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TABLE C3 

Research question 2: What state policies or programs related to formative 
assessment are in place in each Southwest Region state? 



State education agency State education agency 

State regulations, codes, and laws State education agency policies and programs policies on released test items 



Arkansas 



House Bill 2253, filed on 
February 28, 2007, called for a 
two-year pilot using formative 
assessments statewide and 
for the allocation of $25 
per student (though not 
foundation funding) for this 
pilot. The bill, which was 
not introduced through the 
Arkansas Department of 
Education, has been withdrawn 
pending further study 
(interview with department 
representative, April 2007). 



According to the Arkansas Department of 
Education's Rules Governing the Arkansas 
Comprehensive Testing, Assessment, and 
Accountability Program, "The Academic 
Improvement Plan shall include formative 
assessment strategies and shall be revised 
periodically based on results from the 
formative assessments" (Arkansas Department 
of Education, 2006c, 7.02.4, p. 14). The Rules 
authorizes the use of criterion-referenced tests 
to develop individual academic improvement 
plans for students who fail to demonstrate 
proficiency in reading, writing, and math 
(7.02.4). 



Items released on a state 
web site can be used for 
formative purposes (Arkansas 
Department of Education, 
2006b). 



Louisiana 



In 1999 Act 999 required 
the development of student 
academic improvement 
plans for students who fail to 
demonstrate proficiency in 
reading, writing, and math, 
using results from Progress 
of Learning Strategies. The 
Arkansas Department of 
Education Rules Governing 
the Arkansas Comprehensive 
Testing, Assessment, and 
Accountability Program 
(Arkansas Department of 
Education, 2006c, 7.02.4, p. 14) 
requires formative assessment 
strategies to be "included and 
revised periodically." 

Louisiana Administrative 
Code, Bulletin 118, establishes 
a comprehensive assessment 
system (2006). 



According to the Arkansas Department of 
Education's Rules Governing Professional 
Development, "Learning Teams... are to develop 
common formative assessments" (Arkansas 
Department of Education, 2005, 3.05, p.l). 

A report to the Legislative Joint Committee 
on Educational Facilities provides support 
for the development of an online formative 
assessment system, allowing educators, 
students, and parents to monitor student 
progress through test items linked to state 
standards (Technology in Education Task Force, 

2004). 

Smart Start (K— 4), Smart Step (5-8), and Next 
Step (high school) are administered through 
the department to monitor student progress 
(Arkansas Department of Education, 2006a). 

A web site is provided with 
downloadable items for 
practice testing (LEAP and 
GEE); Louisiana Department 
of Education http://www.doe. 
state, la. us/lde/saa/760.html). 

See Practice Assessment/ 
Strengthen Skills (PASS) 
program described below, 
under table C5. 



Key components 4 and 2.c.x. of the Louisiana 
Department of Education's Louisiana Literacy 
Plan call for ongoing formative assessment 
(Louisiana Department of Education, 2006b). 

In response to teachers' requests for 
more finely grained reporting of student 
performance (substrand level) to make 
meaningful data-driven instructional decisions, 
the department secured a grant to develop 
Enhanced Assessments of Grade-Level 
Expectations (EAGLE). The state saw this as an 
opportunity to incorporate current research 
on formative assessment, making the online 
tool serve multiple purposes. The state hopes 
to match timelines with other initiatives, such 
as end-of-course tests, with the goal of being 
operational in May 2007 (Chou 2006; interview 
with Louisiana Department of Education 
project director Ron May, March 13, 2007). 





APPENDIX C 



25 



TABLE C3 (CONTINUED) 

Research question 2: What state policies or programs related to formative 
assessment are in place in each Southwest Region state? 




Louisiana 

(continued) 



This new formative assessment system 
gives teachers access to an online pool of 



items — approximately five per grade-level 
expectation (GLE) — aligned to K-12 standards/ 
GLEs in English and language arts, math, 
science, and social studies. It provides detailed 
information about student learning in an 
on-demand environment. Used diagnostically 
before instruction to develop or modify 
lesson plans during instruction or to evaluate 
student competency after an instructional 
unit, EAGLE supplies GLE-aligned items in a 
vendor-developed item bank. Teachers select 
items to create their own tests, or use pre-made 
tests, and then develop tutorials. Computer- 
automated scoring of multiple-choice items 
(MCs) is combined with hand scoring of some 
short answer items and compositions. With 
immediate feedback on MCs, students may 
take tests, check their scores, and practice 
new skills. Teachers are trained in hand- 
scoring criterion-referenced tests and can 
use EAGLE to deliver information to students 
(such as feedback on homework). Student self 
assessment and goal setting are encouraged 
through guided feedback, helping students 
identify sources of error and types of mistakes, 
and develop new learning skills to reach goals. 
The system provides four reporting options 
for students and six for teachers. It is linked 
to instructional resources for teachers and 
professional development opportunities for 
teachers and administrators. 

Practice Assessment/ Strengthen Skills (PASS), 
developed with Pacific Metrics, provides online 
practice tests in English and language arts, 
math, science, and social studies (print version 
also available). Students learn general test- 
taking skills. PASS gives them an opportunity 
to practice skills assessed on the Louisiana 
Educational Assessment Program (LEAP) 21 and 
graduation exit exam 21. There are elementary, 
middle, and high school versions. Students 
can get instructional feedback geared to 
their responses. PASS is based on formative- 
assessment research (Boston, 2002 and others) 
on the effectiveness of feedback-driven 
integration of instruction and assessment. PASS 
can be used by individuals or a group, at home 
or at school. Teachers cannot access scores 
(Louisiana Department of Education, nd). 



(CONTINUED) 
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TABLE C3 (CONTINUED) 

Research question 2: What state policies or programs related to formative 
assessment are in place in each Southwest Region state? 



State education agency State education agency 

State regulations, codes, and laws State education agency policies and programs policies on released test items 



New 

Mexico 



Reading pre- and post- The New Mexico Public Education Department 

instruction tests for convened a taskforce (department specialists 

kindergarteners are provided and district representatives) to develop a 
through the New Mexico consumer guide for formative assessments to 

Reading Excellence Act (2000). guide districts in decisionmaking as they review 

vendor materials (New Mexico Public Education 
Department, 2006b). The New Mexico 
Consumer Guide reviews literature on formative 
assessment and shelf products. Fifteen criteria 
are used to evaluate product strengths and 
limitations. Vendors are encouraged to submit 
products for review. Districts can also request 
that certain products be reviewed. The 
department endorses the use of the Consumer 
Guide to facilitate professional development 
and linkage to state standards. However, 
they are seen as an interim measure to help 
districts until state formative assessments can 
be developed (New Mexico Public Education 
Department, 2006b; Maple, 2006). 



Memo from Public Education 
Department notifying 
districts of released items 
(Garcia, 2005). 

Released items available on 
state web site (New Mexico 
Public Education Department, 
2006a). 



Oklahoma Oklahoma School Law, Article 
V (Oklahoma School Testing 
Program Act) and Oklahoma 
Administrative Code 210:10-13 
(2003 Revisions): Section 
889 support diagnostic 
testing, in association with 
the Reading Sufficiency Act 
and Reading Proficiency Act 
(Oklahoma State Department 
of Education, nd-d; Oklahoma 
Legislature, nd). 



In 2007 the department filed a request for 
proposals for the development of a voluntary 
state curriculum, with lesson plans and 
curriculum aligned to the American Diploma 
Project and National Assessment of Educational 
Progress (math in 2007/08, reading and science 
in 2008/09). The move is intended to support 
local formative assessment efforts (interview 
with Dr. Karen Harvey, April 2007). 

Oklahoma School Law, Article 
V (Oklahoma School Testing 
Program Act) and Oklahoma 
Administrative Code 210:10-13 
(2003 Revisions): Section 888 
provide sample test items 
for formative use (Oklahoma 
State Department of 
Education, nd-d). 

Teachers can access items for 
use in creating practice tests 
(Oklahoma State Department 
of Education, nd-c). 



Data from end-of-course exams are reported 
to districts. The data can be used diagnostically 
at the student level and formatively to guide 
instruction and remediation (Oklahoma State 
Department of Education, nd-c). 
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TABLE C3 (CONTINUED) 

Research question 2: What state policies or programs related to formative 
assessment are in place in each Southwest Region state? 



State education agency State education agency 

State regulations, codes, and laws State education agency policies and programs policies on released test items 



Texas 



Texas Administrative Code, 
title 19, part 2, chapter 101. 

Texas Education Code, subtitle 
H, chapter 39, subchapter B. 

The Student Success Initiative 
was created by the Texas 
legislature to ensure that 
all students receive the 
instruction and support 
needed to succeed in reading 
and math. The initiative 
provides support for students 
at risk of retention (those 
who failed to pass the Texas 
Assessment of Knowledge 
and Skills [TAKS] at grades 3, 

5, and 8). Grade-Placement 
Committees use formative 
assessments to develop 
personal education plans for 
students (Texas Education 
Agency, 2007b). 



Texas Education Agency, Commissioner's 
Ruling in support of Student Success Initiative: 
subchapter BB of chapter 101 of Texas 
Administrative Code, title 19, part 2 (Texas 
Education Agency, 2007b). 

The Technology Immersion Pilot (TIP), a tool for 
administering diagnostic assessments, delivers 
on-demand data about student proficiency 
for customizing instruction and implementing 
interventions. Aligned with state standards, TIP 
can be used individually or by a group Texas 
Education Agency, 2006b). 

Anita Givens (the agency's director for 
instructional materials and education 
technology) administered a successful pilot 
using handhelds for formative assessment of 
early reading (Texas Education Agency, 2004). 



Section 101.33 of the Texas 
Administration Code states 
that online or hard copies of 
released items can be used 
formatively (Texas Education 
Agency, nd-b). 

Online practice tests for end- 
of-course tests and released 
items from TAKS and Texas 
Assessment of Academic 
Skills (TAAS) can be used 
formatively (Texas Education 
Agency, 2006a). 
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TABLE C4 

Research question 3: What types of support link state policies with district formative assessment practices? 



State 



Professional development opportunities 



Resources 



Product endorsements 



Arkansas 



Professional development time can include 
"developing assessments for learning (formative 
assessments)," according to the Arkansas 
Department of Education's Rules Governing 
Professional Development (Arkansas Department of 
Education, 2005, 5.05.03, p. 7). 

Formative assessment is an approved topic for 
professional development, so that teachers can 
learn to develop, interpret, and use formative 
assessments in each subject area (Arkansas 
Department of Education, 2005). 

The department supports the use of Positive 
Behavior Support funds for professional 
development. A federal grant provides training 
for educators to use student discipline data 
formatively as intervention for students with 
behavioral issues (James, 2004). 



The National Office for 
Research on Measurement 
and Evaluation Systems 
(University of Arkansas) 
provides the Enterprise 
Guide for Educators, giving 
state teachers access 
to data "for use in the 
improvement of classroom 
instruction" (National Office 
for Research, Measurement, 
and Evaluation, 2007). 



TheTriand web-based 
system is used to manage 
data associated with district 
formative testing (Arkansas 
Department of Education, 
nd). 



The department supports staff development 
in using data derived from the Arkansas Target 
Testing Initiative (see table C5) through Arkansas 
Education Service Cooperatives (South Central 
Service Cooperative, nd). 



Training for teachers (K— 3 and special education) 
uses the results from Dynamic Indicators of 
Early Literacy Success (DIBELS) formatively in 
developing individualized intervention plans 
(Arkansas Department of Education, 2006d). 



Louisiana 



The Educational Testing Service offers professional 
development (Webinars) to teachers using a 
Formative Assessment Item Bank (Educational 
Testing Service, 2007). 

Professional development is provided for all 
districts in using the state's formative assessment 
tool, Enhanced Assessments of Grade-Level 
Expectations (EAGLE, see table C3). 

The state provides a Practice Assessment/ 
Strengthen Skills (PASS) Teacher's Guide, 
instructional brochures, and a Help Desk (PASS, 
see table C3). 

The Educational Testing Service offers professional 
development (Webinars) to teachers using a 
Formative Assessment Item Bank (Educational 
Testing Service, 2007). 



After Katrina, the American 
Education Corp (AEC, 
based in Oklahoma) 
offered free access to 
online assessments for 
displaced K— 1 2 students 
so that receiving schools 
have a diagnostic tool for 
judging student skill levels. 
Formative assessments are 
provided in math, reading, 
science, and social studies 
(American Education 
Corporation, 2005). 



No evidence. 



EAGLE, a formative 
assessment tool, is 
provided to all districts (see 
table C3). 
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TABLE C4 (CONTINUED) 

Research question 3: What types of support link state policies with district formative assessment practices? 



1 State 


Professional development opportunities 


Resources 


Product endorsements 


New 

Mexico 


No evidence. 


The New Mexico Consumer 
Guide reviews the literature 
on formative assessment 
and shelf products (see 
table C3). 


No evidence. 


Oklahoma 


The Educational Testing Service offers professional 
development (Webinars) to teachers using its 
Formative Assessment Item Bank (Educational 
Testing Service, 2007). 


No evidence. 


The Oklahoma State 
Department of Education 
endorses a product called 
Data Connections: Using 
Assessment to Improve 
Teaching and Learning 
(Appalachia Education 
Laboratory and Edvantia, 
2003). 


Texas 


The Technology Immersion Pilot offers a 
professional development component and 
technical support (see table C3; Texas Education 
Agency, 2006b). 

The Educational Testing Service offers professional 
development (Webinars) to teachers using its 
Formative Assessment Item Bank (Educational 
Testing Service, 2007). 


The Technology 
Immersion Pilot is a tool 
for administering online 
diagnostic assessments (see 
table C3; Texas Education 
Agency, 2006b). 


Through the state's Student 
Success Initiative, districts 
can use Pearson's Progress 
Assessment Series (PAS), an 
online formative assessment 
series based on lexiles 
(reading) and quantiles 
(math) and "designed to 
guide instructional support 
decisions when used 
diagnostically." PAS is linked 
to Texas standards (Pearson, 
2007b). 
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TABLE C5 

Research question 4: What examples of district-initiated formative assessment 
practices can be identified in each Southwest Region state? 



State Examples 



| State 


Examples 


Arkansas 


Ten Arkansas Education Services Cooperatives are collaborating on the Arkansas Target Testing Initiative, a 
formative assessment initiative that offers schools access to four tests with items "very similar to the items 
included in the state benchmark and end of level exams." The items have been developed to help teachers focus 
on student strengths and weaknesses prior to the state exams. The testing initiative also provides schools with 
instructional pacing guides for each content area (South Central Service Cooperative, nd). 

Princeton Review shelf tests are used formatively in Little Rock. 

The Educational Testing Service Formative Assessment Item Bank is used in some districts (Educational Testing 
Service, 2007). 

Smart Tutor, a web-based system for assessment-driven differentiated instruction, is used diagnostically in some 
high-risk districts (Learning Today, 2006). 

Many districts have comments about using data formatively in handbooks for teachers. 

Dynamic Indicators of Early Literacy Success (DIBELS) is used in primary grades in many districts (Arkansas 
Department of Education, 2006d). 

The Arkansas Learning Management System is a community of schools that shares instructional materials, 
including test items for formative use (Arkansas Learning Management System, 2004). 

According to the National Center for Educational Accountability, successful middle schools in Arkansas are using 
formative assessment to monitor student progress and evaluate instruction. The schools support "landmark" 
testing (National Center for Educational Accountability, 2006). 


Louisiana 


The Educational Testing Service Formative Assessment Item Bank is used in some districts (Educational Testing 
Service, 2007). 

According to the National Center for Educational Accountability, schools from five parishes received awards for 
best practices because they use regularly collected data, the PASS system, or a number of diagnostic tools to 
guide curricular and instructional adjustments, leading to higher performance data (Just for the Kids, Louisiana 
Elementary Best Practice Study 2004-05 (National Center for Educational Accountability, 2006). 

The Union Parish School District uses Pearson's Classroom Performance System (CPS), a hand-held device for 
tracking feedback, to monitor student learning (Pearson, 2007a). 


New 

Mexico 


One district (Las Cruces Public Schools) developed a comprehensive online Curriculum Alignment Matrix 
and posts links to formative assessment items for each grade and subject. Formative assessment is explicitly 
incorporated in the Curriculum Alignment Matrix, with web tools that guide schools in combining pacing guides, 
state test results, and local formative assessment initiatives to raise student achievement (Las Cruces Public 
Schools, 2006; personal communication with district representative, May 2007). 

Some districts are using ThinkLinkfor periodic online testing that is benchmark-based (correlated with state 
standards). Teachers also can create their own testlets from their item banks (Discovery Education, nd). 

According to the National Center for Educational Accountability, some award-winning districts used packaged 
instructional programs (purchased with local monies) that included formative assessments. Teachers commented 
that they like it when formative assessments look like the state test because then weekly tests look like end- 
of-year tests. Many different assessments are used by others to help identify competencies in which students 
require further instruction and individual students who are falling behind and need tutoring. Others use 
formative assessment results to modify instruction (National Center for Educational Accountability, 2006). 
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TABLE C5 (CONTINUED) 

Research question 4: What examples of district-initiated formative assessment 
practices can be identified in each Southwest Region state? 



State Examples 



| State 


Examples 


Oklahoma 


Some districts developed online systems, such as a Student Information System, to monitor formative 
assessment data. 

The Educational Testing Service Formative Assessment Item Bank is used in some districts (Educational Testing 
Service 2007). 

Scantron formative and diagnostic programs are used in some districts (Scantron Corporation, 2007). 

Many districts use diagnostic testing to assess the appropriateness of special education for struggling students. 

According to the National Center for Educational Accountability, successful middle schools in Oklahoma are 
using multiple tests (such as district-developed, standards-aligned, criterion-referenced tests or the Stanford 
Achievement Test-10) formatively to monitor individual student progress and evaluate instruction. Some 
are using monthly benchmark assessments combined with observation (National Center for Educational 
Accountability, 2006). 

The Jenks Public Schools were awarded a national quality award for ongoing use of formative assessment data. 
They use preVpost-tests and Essential Skills (National Institute of Standards and Technology, 2007; personal 
communication with district representative, May 2007). 


Texas 


The Educational Testing Service Formative Assessment Item Bank is used in some districts (Educational Testing 
Service, 2007). 

The Clarksville Independent School District is using a combination of the Technology Immersion Pilot (Texas 
Education Agency, 2006b) and CTB/McGraw-Fiill's i-Know program (CTB/McGraw-Flill, 2007). 

According to the National Center for Educational Accountability, three high schools were found to be award- 
winning, using pyramids of interventions based on formative and summative data and providing professional 
development related to formative assessment (National Center for Educational Accountability, 2006). 
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APPENDIX D 

LETTER OF INTRODUCTION TO 
DISTRICT SUPERINTENDENTS 

May 25, 2007 UUesti-d ' 



Superintendent 
District Name 
District Address 
District City, Zip 



Dear [Superintendent]: 

As part of a research study we are conducting on behalf of the Regional Educational Laboratory Southwest 
(REL-SW), your district was identified as one that has implemented a formative assessment initiative. The re- 
search study focuses on formative assessment initiatives that have been implemented in each state in the region 
(Arkansas, Louisiana, New Mexico, Oklahoma, and Texas). We are writing to request your permission to speak via 
telephone with a district representative who is knowledgeable about your district’s formative assessment initiatives. 
This brief (10-15 minutes) telephone interview is intended to provide information to supplement the data that were 
collected from state and district documents and Web sites, as well as through interviews with state-level personnel. 

At your earliest convenience, we would appreciate your forwarding to us the name and contact information for 
your formative assessment designee. Alternatively, the designee may contact us directly via the telephone numbers 
or email addresses provided below. We will be available to speak with your designated representative at his or her 
convenience. 

We appreciate your cooperation with this important research effort. As a token of our appreciation, we will ensure 
that your district receives a copy of the final report following its approval by the U.S. Department of Education, 
Institute of Education Sciences. 



Sincerely, 



Carole Gallagher, Ph.D. 

415-615-3211 

cgallag@wested.org 

Peter Worth 

415-615-3336 

pworth@wested.org 
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APPENDIX E 

QUESTIONS FOR DISTRICT REPRESENTATIVES 
RESPONDING BY EMAIL 



Interview Goals 

It is hoped that responses to these questions will 
support the formative assessment research study 
by contributing to a more comprehensive under- 
standing of the ways in which formative assess- 
ment initiatives are implemented at the district 
level and by clarifying information about this 
initiative for the final report. 



Questions 

1. Please describe your district’s formative as- 
sessment initiative. 

2. What was your district’s rationale for imple- 
menting this initiative? What do you hope to 
accomplish? How are formative assessment 



data currently used at the school and district 
levels? 

3. In what ways has the state guided or sup- 
ported the development of formative assess- 
ment practices in the district? What resources, 
financial or otherwise, have been provided? 
What training or professional development 
was provided? Does the state monitor imple- 
mentation of this program in your district? 

4. What have been the effects of this forma- 
tive assessment initiative? Have you received 
feedback from the state, schools, or other 
sources on the effectiveness of the formative 
assessment initiative you implemented? What 
is working? How is it helpful at the school and 
district levels? 

5. What could be improved in your district’s 
formative assessment initiative? What would 
need to happen for that improvement to occur? 
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